It is common to analyze large data sets in the process of making business decisions. Such data sets may be thought of as comprising a dimensionally-modeled fact collection. For example, each “record” of the fact collection may represent attributes of an “entity” or “item” such as a particular user of online services, whereas the value in each field of the record represents a value of a particular characteristic of that entity (e.g., age of user, gender of user, number of online page views by that user, etc.). While the term “record” is used, this is not meant to imply any particular organization of the dimensionally-modeled fact collection (e.g., such that the fact collection is organized as a relational database, etc.).
In many situations, the data of a data set resides in a persistant storage device, such as a file system, data warehouse or database. The data of the data set is loaded into the memory of a computer system in order to process the data using an analysis application being executed by the computer system. For example, FIG. 1 illustrates a system 100 in which a computing device 102 is connectable via a network 104 to a server 112. An application being executed by the computing device system (e.g., by a processor or processor system 110) may interoperate with the server 112 to cause data to be imported from a data store 106 into memory 108 of the computing device to be processed. The data store 106 may be directly connected to the network 104 (e.g., a SAN device) or directly attached to the computing device (such as a hard drive of a personal computer).
An example of an analysis application includes an application to visually explore the data. If the data set is too large to fit in the available space of the memory, the application may fail due to memory availability restrictions. The failure may become apparent only after a long attempt import process. In some other cases, as the analysis application executes, the data may be swapped into and out of memory from a slower storage, such as from a disc, and as a result, the application may perform poorly.